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B. Cessac and J. A. Sepulchre 
Institut Non Lineaire de Nice, 1361 Route des Lucioles, 06560 Valbonne, France 

We discuss the abiUty of a network with non linear relays and chaotic dynamics to transmit 
signals, on the basis of a linear response theory developed by Ruelle [J for dissipative systems. 
We show in particular how the dynamics interfere with the graph topology to produce an effective 
transmission network, whose topology depends on the signal, and cannot be directly read on the 
"wired" network. This leads one to reconsider notions such as "hubs". Then, we show examples 
where, with a suitable choice of the carrier frequency (resonance), one can transmit a signal from a 
node to another one by amplitude modulation, in spite of chaos. Also, we give an example where 
a signal, transmitted to any node via different paths, can only be recovered by a couple of specific 
nodes. This opens the possibility for encoding data in a way such that the recovery of the signal 
requires the knowledge of the carrier frequency and can be performed only at some specific node. 
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Common sense contrasts chaotic dynamics to coherent dynamics. For example, considering a network 
of dynamical units having a collective chaotic dynamics, it looks unlikely that an efficient transmission 
of a coherent signal between two arbitrary units of this system would be feasible, since the "butterfly 
efltect" would scramble and wash out any coherent signal that could be injected in the system. On the 
other hand, there is also a common intuition that coherence can emerge through appropriate averaging 
procedures. This is one of the cornerstones of statistical physics which succeeds in explaining some 
"simple" behavior of macroscopic systems emerging from the "complex" behavior of its microscopic 
constituents. 

Recently, new advances have shown that some fundamental results in non equilibrium statistical 
mechanics, such as linear response theory, may be extended and generalized to dissipative chaotic 
systems [Ij. "Dissipative" means here that the long term dynamics occurs on an attractor in phase 
space, a feature which is absent in conservative physical systems. In a previous paper [2] we have 
shown that this theory can be applied to analyze networks of dynamical units having a chaotic and 
dissipative dynamics. In our case, the "dissipation" comes from saturation efl"ects in the transfer 
function of the units. This feature is known to be widespread in biological networks and also occurs 
in communication networks. The main result of our first study was to show the existence of some 
resonances, called "stable" resonances, predicted in and proper to dissipative chaotic dynamics. 

In the present paper we show that such dynamical networks can behave as a selective input/output 
system. We demonstrate that a network of randomly interconnected units exhibiting collective chaos, 
combined with a "natural" averaging procedure, can be used as an open system in which a weak 
coherent signal can be sent from one node and recovered by another receiver node. It is shown 
that such transmission possesses a selectivity, in frequency and in the receiver node, due to the 
aforementioned resonances. These propagation phenomena are illustrated on a particular example, 
but they are theoretically predicted in the firm framework of the linear response theory. Therefore 
our findings are not specific to this example. Moreover, the method introduced here, relying on 
the computation and the analysis of a complex susceptibility, can be easily applied to other systems. 
Indeed, stable resonances cannot be found by studying correlations functions. Consequently, the 
approach proposed here goes beyond the simple correlation analysis and brings relevant additional 
informations. 

Besides the possible use of our results in natural or in artificial networks, our conclusions point out 
that the study of "networks", which has recently flourished, should not only focus on the topology 
of the graph associated with these networks, but also on its dynamics. If an interconnected network 
is composed of dynamical units, analyzing the collective emergent dynamics is essential to assess the 
notion of effective "hub", or of eff"ective "connectivity" in order to characterise the ability of this 
system to transfer information. 
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The use of statistical approaches, and especially, statistical mechanics, has been quite successful in many research 
fields, sometimes far away from physics. A recent prominent example is the study of "networks" . There is indeed a 
widespread activity in this domain and many beautiful and unexpected results have for example been derived about 
the topological properties of random graphs, scale free graphs, small world, etc.. [^^|^|^. Certainly, these studies 
are useful to understand and anticipate the behavior of communication networks such as Internet or mobile phones, 
social networks, transport networks, epidemic propagation on networks H^j, etc... 

However, one can raise two objections to these approaches. First, a large part of these studies focuses on topological 
properties of the underlying graph and the nodes, whatever they represent, are considered as "neutral" entities. 
However, in many cases, the nodes are active nodes behaving in a nonlinear way. For example, in a communication 
network a relay regenerates (amplifies) weak signals, but it has a finite capacity and saturates if too many signals 
arrive simultaneously. A neuron has a nonlinear response to an input current, a gene expression is determined by 
a nonlinear function of the regulatory proteins concentration, etc.. These nonlinearities might modify the network 
capabilities in a drastic way. This suggests that the mere study of the graph topological structure of a network with 
nonlinear nodes is not sufficient to capture the full dynamical behavior. 

The second objection concerns the statistical approach itself. Indeed, statistical results do not deal with individuals 
characteristics. But one might be interested in knowing the characteristics of the network he is currently studying 
or using. For example, somebody using his mobile phone is not so much interested in the general properties of the 
class of general graphs including mobile phones networks. Rather, he is interested in the transmission ability of the 
network that he is currently using, with its particularities. In particular, it is not guaranteed that all nodes of this 
network behave in the same way, especially if they interact nonlinear ly, with possible excitation/inhibition effects 
and asymmetric interactions. Actually, the study of biological networks such as neural networks or genetic networks 
suggests exactly the opposite. The emerging dynamics may induce a differentiation in the effective role of each node 
in the global dynamics, and the nodes are usually not interchangeable. What happens if one perturbs one or another 
node in this network ? Are the effects the same ? How well is a "signal" sent by a given node received by another 
one? One may expect that the answer depends on the pair sender /receiver. Statistical resultsj2|i will wash out the 
particularities of each network realisation. 

A careful investigation of these questions requires therefore a proper analysis of the dynamical system corresponding 
to the network currently used and may require the development of new tools and methods. In some situations, 
statistical mechanics still provides useful insight. But one has to "adapt" it to the present situation and to forget 
somewhat the conventional wisdom. Thus, in a recent paper 0, we have shown that the linear response theory, 
developed by Ruelle |3] for dissipative dynamical systems, can be efficiently used to characterize the behavior of a 
network where the nodes have nonlinear transfer functions. That paper focused essentially on the existence of a new 
type of resonances, called stable resonances, predicted by Ruelle but never observed before. These resonances only 
exist in systems where the volume is dynamically contracted in the phase space and they are not observed in the 
power spectrum. In our case, this contraction came from saturation effects in the transfer functions. 

In the present paper, we focus on potential applications relying on the existence of such resonances. We are mainly 
interested in the ability of such a network with a chaotic dynamics to propagate a signal. On the basis of the linear 
response theory, we first conjecture, in section some non-intuitive and unexpected properties. We then exhibit an 
example supporting these conjectures in the section^ First, we compute the Fourier transform of the linear response, 
called the complex susceptibility, and show how the presence of stable resonances may violate the conventional non 
equilibrium statistical mechanics wisdom about linear response, fluctuation-dissipation theorem, and relaxation to 
equilibrium (sections III Bill Many of these ideas are already in Ruelle's work, but their interpretation in the 
context of nonlinear networks is new. We then argue that it is indeed possible to transmit and recover a signal in a 
chaotic system. We show, in particular, how the dynamics interfere with the graph topology to produce an effective 
transmission network, whose topology depends on the signal and cannot be directly read on the "wired" network. 
This leads one to reconsider notions such as "hubs" (section III Dp . Then, we show examples where, with a suitable 
choice of the resonance frequency, one can transmit a signal from a node to another one by amplitude modulation, 
in spite of the presence of chaos (section III Fp . In addition, we give an example where a signal, transmitted to any 
node via different paths, can be performed only at some specific node. Finally, we discuss the effect of increasing the 
signal amplitude, going beyond linear response in section FlI Gl 

I. GENERAL SETTING AND PURPOSES. 

In this section we recall the main results established in and state the questions addressed in the present paper. 
Consider a set of N nodes (or relays) connected on a graph. The link from j to i is denoted by Jij. Links are oriented 
{Jij ^ Jji) and signed. The sign mimics excitation/inhibition effects. These effects are obviously present in biological 
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network but they can also exist in communication networks. For example, regulation systems exist, designed to 
optimize the bandwidth capacity. These systems can balance the activity of a given relay with another one, resulting 
in an effective excitation/inhibition. Note that, in our model, the J^'s do not depend on the state of the node, but 
the linear response theory accommodates this generalisation. A zero link means that there is no connection from j 
to i. In the sequel the Jy's are fixed and do not evolve in time. Moreover, as argued in the introduction, we are 
interested in the behavior of a network having a fixed set of Jij 's and we do not consider statistical results relying on 
averages over some probability distribution for the Jy 's. 

The activity of a node i is characterized by a continuous variable Xi. It is determined by the set of signals coming 
from the nodes connected to i, the signal coming from j being weighted by Jij. Denote by Ui{t) the total input received 
by i at time t. This is a function of the Xj^s and of the 's. We assume that the activity of i evolves according to 
Xi{t + 1) = f{ui{t)), where f{x) is a sigmoidal transfer function with a slope g (e.g. f{x) — tanh{gx)) . Moreover, 
we consider the case where 17 >> 1, namely the sigmoid is strongly nonlinear. Note that sigmoid transfer functions 
are encountered in neural networks (when the neuron activity is described in terms of frequency rates), in genetic 
networks (Hille function), and they may also be suitably represent amplification/saturation effects in communication 
protocols such as TCP/IP. This amplification/saturation effect is actually the most important characteristics in what 
follows. 

Assume indeed that we superimpose to the "background" input Uiit) of the node i a small signal S,i{t). How does 
this signal propagates inside the network ? Because of the sigmoidal shape of the transfer functions the answer 
depends crucially on the activity of the nodes. Assume, for the moment and for simplicity, that the time-dependent 
signal ^i{t) has variations substantially faster than the variations of Ui. Consider then the cases depicted in Fig. njL,b. 
In the first case the signal ^i(t) is amplified by /, without distortion if ^^(t) is weak enough. On the other hand, it 
is damped and distorted by wild nonlinear effects due to saturation, in Fig. ^p. This example shows that the signal 
propagation in such a network must take into account the topological structure of the graph as well as the nonlinear 
effects. This simple remark leads one to reconsider the notion of "hub" . A hub is a relay with a strong connectivity. 
From a topological point of view, this is certainly a very important node. But, when considering signal propagation 
in networks with saturating relays, the role of a hub might be temporarily weakened if this hub is maintained in a 
saturated state by the global activity. 




FIG. 1: nonlinear effects induced by a transfer function with a sigmoidal shape on signal transmission. Fig. Amplification. 
Fig. ^p. Saturation. 

In some situations, it is possible to analyze the combined effects of topology and nonlinearity on the propagation 
of weak amplitude signals. This is the goal of the method developed in On technical grounds one first needs to 
make the assumption that the global spontaneous activity is chaotic. The relevance of chaos for realistic situations 
may be debated (note however that chaos generically occurs in the model presented below) and we deal with this 
point in the discussion. But, at the present stage, our point is slightly different. On one hand we provide an example 
where the combined effects of topology and nonlinearity can be handled, and, on the other hand, we establish that 
one can study the propagation and the effects of a signal superimposed upon the chaotic background in spite of (and 
in fact thanks to) chaos. 

To be more specific consider the following model. The input signal Ui(t) is a function of the activity Xj(t) of the 
units j connected to i and it is given by Ui{t) ~ J2j Jij^ji^)- Then the global dynamics writes: 



u(t + l)-G[u(t)] = J./(u(t)), 



(1) 
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where u{t) = and where we used the notation f{u{t)) = {f{ui{t))}-^-^. J is the matrix of couphng 

coefficients. 

The saturation of the sigmoid discussed above (fig. ^p) has the dynamical effect of producing volume contraction in 
the phase space. Indeed, the Jacobian matrix is given by DGu = J'A(u), where A is a diagonal matrix with Aii(u) — 
f'{ui), and its determinant is given by detZ'G'u = det(J') x Y[i=i f (''^i) ■ The determinant has an absolute value 
strictly lower than 1 provided that some Mi's are strong enough (corresponding to a saturation of the corresponding 
unit). 

Consequently, the asymptotic dynamics settle onto an attractor. This attractor is generically unique (provided that 
one breaks the symmetry u — u of the transfer function f{x) — tanh{gx), with, e.g. a small time independent 
threshold added to the local field u). 

The dynamics generically exhibits a transition to chaos by quasi-periodicity as g increases, for suitable choices 
of the Jij's (e.g. independent, identically distributed random variables with scaled mean and variance 1^ ITollTl| ) . 
[Recall however that the Jy 's are fixed during the evolution and that we consider a specific realization of the J^'s 
(we do not average over the disorder)]. Thus the dynamics asymptotically settles onto a chaotic attractor provided 
that g is sufficiently large. The statistical properties of the dynamics on its attractor are characterized by the Sinai- 
Ruelle-Bowen measure p (SRB) which is obtained as the weak limit of the Lebesgue measure p, under the dynamical 
evolution: 



In the following we will assume that all Lyapunov exponents are bounded away from zero (weak hyperbolicity). 
Then for each u G suppp, where suppp is the support of p, there exists a splitting E'^ (u) (B E"^ (u) such that i?"(u), the 
unstable space, is locally tangent to the attractor (the local unstable manifold) and E''{u.), the stable space, is trans- 
verse to the attractor (locally tangent to the local stable manifold). Let us emphasize that the stable and unstable 
spaces depend on u (while the Lyapunov exponents are p almost surely constant). Let us consider a point u on the 
attractor and make a small perturbation Su- This perturbation can be decomposed as = 5^ + i5u where 5^ G £'"(u) 
and (5„ G E^{u). is locally amplified with an exponential rate (given by the largest positive Lyapunov expo- 
nent). On the other hand is damped with an exponential speed (given by the largest negative Lyapunov exponent). 

Assume now that we superimpose a weak signal upon the (chaotic) activity. For simplicity, we shall assume that 
the signal does not depend on the state of the system (linear response still applies in this case, but the equations H4I5() 
do not hold anymore). Denote by ^ the vector The new dynamical system is: 



The weak signal ^{t) may be viewed as small perturbation of the trajectories of the unperturbed system 
Consequently it has a decomposition on the local stable and unstable space. The stable component is exponentially 
damped. The unstable one is amplified by the dynamics and nonlinear terms rapidly scramble and mix the signal. 
Consequently, it becomes soon impossible to distinguish the signal from the chaotic background. 

This is the effect observed on individual trajectories. However, the situation is substantially different if one considers 
the average effect of the signal, the average being performed with respect to the SRB measure p of the unperturbed 
system. It has been established in that the average variation of the local field Ui under the influence of the signal 
is given, to the linear order, by: 



P 



lim G"/i. 



(2) 



u{t + 1) = G [u(<)] + ^(<) = G [u{t) 



(3) 



t 




(4) 



where xi'^) is the matrix : 




(5) 



that writes in explicit form: 




(6) 



We used the shortened notation < > for the average with respect to p. The sum holds on each possible path %j{a), 
of length (T, connecting the unit fco = j to the unit ka- = i, in cr steps. One remarks that each path is weighted by the 
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product of a topological contribution depending only on the weight Jij and of a dynamical contribution. Since / is 
a sigmoid the weight of a path ^ij{u) depends crucially on the state of saturation of the units fco, . . . , k^^i at times 
0, . . . , (T — 1. In particular, if f'{uki_i (/ — 1)) > 1 a signal is amphfied while it is damped if f'{uki_^ {I — 1)) < 1 (see 
Fig. Consequently, though a signal has many choices for going from j to i in cr time steps, some paths may be 
"better" than some others, in the sense that their contribution to Xiji'^) is higher. In particular, this contribution 
depends strongly on the time correlations between the levels of saturation /'(wj), /'(ufe^ ),..., /'(iti) of the units 
j, fci, . . . , i composing the path. 

This observation leads us to several remarks. 

1. The paths j — > i are a priori not equivalent. They have a different weight that depends on one hand on the 
topological contribution and on the other hand on the nonlinearity of the transfer function (this last effect indeed 
does not exists for linear transfer functions). 

2. The average effect of the signal, measured at time i, is a sum of a large number of contributions, resulting from 
the various possible paths, with time delayed versions of the signal. Assuming for example that ^ is periodic, 
the observed effect is that of a sum of waves with different amplitudes and delays. The global effect can be weak 
if the waves interfere in a destructive way, or strong if they interfere in a constructive way. This suggests that 
resonances may occur. If one computes the Fourier transform of Xij{i)-i denoted by Xij(^) and called complex 
susceptibility, one observes resonance peaks corresponding to poles in the complex plane (see 0). An example 
is given in the section III Bl 

3. A natural (physicists) reflex would be to seek these resonance in the Fourier transform of the correlation function 
Cij{t) = {ui{t)uj{Q)) — {ui) {uj) of the pair ij. Indeed, the fluctuation-dissipation theorem basically tells us that a 
susceptibility is (the Fourier transform of) a correlation function. This is true for dynamical systems encountered 
in physics, where the (microscopic) dynamics preserves the volume in the phase space (Liouville theorem). But 
this is no longer true in our case where the dynamics contracts the phase space volume. Actually, the complex 
susceptibility contains more information than the power spectrum. 

Indeed, the Jacobian matrix, as we saw, can be split into 2 parts, corresponding to the action of G in the local 
stable and unstable space, respectively. This means that the response function Q (the corresponding suscepti- 
bility) decomposes in a stable and an unstable part. Each part has its resonances and they can be drastically 
different. On the one hand it has been shown by Ruelle that the unstable contribution is actually a correla- 
tion function (this is a generalized version of the fluctuation-dissipation theorem). Henceforth the resonances of 
the unstable part are contained in the power spectrum. They are called "Ruelle-Pollicott resonances" |0| and 
they do not depend on the observable (provided the observables belong to the same suitable functional space). 
Practically, in our case, this means that these resonances do not depend on the pair ij |20j . Furthermore, since 
|C'ij(w)| = 1(7^^(0;) I the analysis of these resonances does not tell us which units excites and which unit responds 
(see section III Cll . In this sense the analysis of the correlation does not display causal information (except the 
trivial property Cij(t) — Cji{—t)). But the main drawback of correlations functions is that they do not contain 
all possible resonances. 

Indeed, the stable part displays additional resonances, called in the following stable resonances. They are not 
Ruelle-Pollicott resonances. They may also depend on the pair ij. Indeed, the susceptibility of the pair i ^ j is 
in general distinct from the susceptibility of the pair j —>■ i and they may have distinct resonances (see Fig. |S^). 
Note also that the corresponding response function are causal since the stable directions introduce an arrow of 
time (see Fig. |S|d). Finally, on numerical grounds, the computation of complex susceptibilities affords a better 
resolution frequency than the computation of correlation functions (see section Hi C|) . 

Therefore the complex susceptibilities give us essential information about the average effect of a periodic signal, 
applied by some unit onto some other unit. This is investigated in some details in sections III Bill Fl 

4. The equation (jSJ opens in principle the possibility of inducing a response of i, by exciting j with a suitable 
frequency, even if there is no direct link between the 2 units. On the contrary, there may exist a direct link 
between j and i and, in spite of this, there may not be a measurable effect if the frequency of the excitation 
applied to j does not correspond to a high response of i. This enhances the effect of nonlinearities in the effective 
capacities of the network. This is discussed in the section Hi Bl The notion of "hub" is in particular revisited in 
the section Hi Dl 

5. The existence of the stable part may lead to a violation of the standard wisdom stating that the characteristic 
time of return to equilibrium is equal to the characteristic time for mixing. Actually, stable resonances introduce 
additional time scales that can be relatively longer than the mixing time. (The mixing time is given by the 
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Ruelle-PoUicott resonance that is the closest to the real axis in the complex plane). As a matter of fact, it is 
in principle possible to observe the average effect of a signal corresponding to a kick, over a time substantially 
larger than the correlation time. An example is shown in section III CI 

6. Finally, the existence of resonances opens the possibility of using amplitude modulation to transmit a signal from 
a specific unit to another specific one, in spite of (thanks to) chaos. The original signal is then recovered by a 
suitable averaging procedure. This is discussed in section Hi Fl Let us emphasize that the procedure suggested 
here is not control of chaos. We do not stabilize the dynamics on a periodic orbit by a suitable perturbation. The 
perturbed dynamics stays chaotic and we use some natural properties of chaos, such as mixing, to reconstruct 
any weak signal by a suitable average. 

In the next section, we present an example, based on the model supporting these claims. For this we select a 
specific set of Jij 's, randomly drawn, and we focus on the characteristics of this particular network. We do not perform 
statistical averages on the distribution of the Jij 's. As argued in the introduction we want indeed to provide analysis 
tools allowing an user to extract the characteristics of the network he is currently using. One may however ask about 
the genericity of this example. What happens for a different set of J^ 's ? What happens if one increases the size ? 
The statistical behavior of this model has been widely studied in |^ Q ^3 . It has been shown that chaos generically 
occurs provided g is sufficiently large. The average critical value for the transition to chaos has been analytically 
computed in [Tol [lll | . The thermodynamic limit was also fully characterized for a mean field version. From these 
studies and from the theoretical arguments developed above we claim that the behavior described is generic in this 
model. Another example of resonance curves has been produced in |3| for a fully connected version of the coupling 
matrix. 

Obviously, some features such as the resonance frequencies are specific to the choice of the J^ 's. But this is precisely 
what interests us. Starting from a chaotic network with specific resonances, we are able to compute numerically the 
susceptibilities by suitably exciting the nodes. This procedure does not require an a priori knowledge about the 
dynamics. From the susceptibilities curve we extract the resonances of this network and we then use them for 
applications. 

One may also ask about the genericity of the model itself. This point is examined in the discussion. At this stage 
we simply want to remark that most of the effects exhibited in the next section are predicted from the general theory 
presented above. These effects are non-intuitive and depart widely from the conventional wisdom about chaotic 
systems. They also open new perspectives in the study of networks. This example is therefore designed to check that 
these theoretical predictions can be realized in at least one example. Also, the analysis performed here can be easily 
reproduced in other examples or models, possibly more realistic (see the discussion). 



II. SIGNAL PROPAGATION 



A. Model example 

The numerical simulations presented here have been performed on the following example. The number of units was 
fixed to = 9. The network is sparse. Each unit receives connection from exactly K = 4 other units. The J^^'s 

have been drawn at random according to a Gaussian distribution with mean zero and a variance ^ . This ensures the 
correct normalization of the local fields Uj 8] . The version of the J7 for which all simulations have been performed is : 
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Coupling matrix J' used in the simulations below. 



(Note that the corresponding graph is not decomposable). The corresponding network is drawn in Fig. El Blue 
stars correspond to inhibitory links and red crosses to excitatory links. It is for example easy to see that the unit 7 
is a "hub" in the sense that it sends links to almost every units, while 0, 2, 3 or 5 send at most two links. 
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FIG. 2: Connectivity matrix for the simulations performed in this paper. 



A small constant 9i has been added to each u,; to break down the symmetry u — u (i.e. Ui{t) = ^ JijXj{t) + 0i). 

The corresponding dynamics exhibits a transition to chaos by quasi-periodicity. For g = 3 the clynamics has a 
strange attractor. There is one positive Lyapunov exponent (Ai = 0.153) and 8 negative Lyapunov exponents (with 
A2 = —0.427). Hence the system is weakly hyperbolic (all Lyapunov exponent bounded away from zero). The 
spectrum is stable to small variations of g. The Kaplan- Yorke dimension is 1.64. 



B. Computation of susceptibilities. 

In a nutshell (see for more details) the computation of susceptibilities consists in perturbing the trajectories 
of Q by two perturbations ^^^HO = ee^ cos(wt) and ^'^''(t) = — eej sin(ti;t). If u^^^jU^^^ denote the variables of the 
corresponding perturbed systems then one may write (for lo ^ 0): 

X.,(^)= hm -lf]e-(*-i)[^if' (t)+z^r(t)]. (7) 

1 — >oc 1 6 

t=0 

This provides a straightforward way to compute the susceptibility where most of the computing time goes into 
computing the orbits u^'^^ {t) . From a numerical point of view the precision of Q can be improved by performing an 
additional average over several trajectories. This allows one to compute error bars. 

Note that the average is directly performed on the perturbed trajectories, and not on the difference between the 
perturbed and unperturbed trajectories. This has two consequences. On one hand this avoids to iterate simultaneously 
the perturbed and unperturbed system, compute the difference, and renormalize it when it becomes too large, to keep 
only linear effects. Instead, the computation (|7|) includes all nonlinear effects and these are precisely these effects that 
permit to compute the average with respect to p. Indeed, the SRB measure Q is exactly an average over a typical 
trajectory including all nonlinear effects such as mixing and folding. 

The perturbation ^'•*^(t) decomposes into a stable and unstable part. The unstable part is rapidly amplified by 
the initial condition sensitivity, then nonlinear contributions arise, leading to mixing and to an effective average 
on the attractor, when T 00. This average is the Fourier transform of a correlation function at the frequency 
ui. Since correlation functions decay exponentially in chaotic system p]| the sum Q converges and gives a finite 
contribution to the unstable part. The stable part is damped by contraction, but since it is applied in a continuous 
way, one obtains an effective summation of the effects of a sinusoidal perturbation transverse to the attractor. One 
finally obtains a finite quantity giving the (average) response of the system to a perturbation having projections 
in the stable and unstable directions. When e is small this is the linear response. Note however that there is a 
priori no condition on e in eq. ^ j22j |. This means that Xiji'^) corresponds to the response of the system to the 
perturbation, possibly including nonlinear contributions in e, whenever e is too large. One has therefore to check that 
e is small enough to ensure that Xij does not vary when e varies on a small interval. An example is given in section lll El 
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Some examples of susceptibilities computed in this way are depicted in Fig. Owith e = 10 ^. The computation has 
been done with T — 262144 and 100 trajectories corresponding to have 26.214.400 points for each oj. The frequency 
samphng is 5uj = Jfg. 




FIG. 3: . Modulus of some susceptibilities. Fig. |^. 7 (highly connected unit) excites the units: 1 (excitatory link with intensity 
Ji7 = 0.007); 2 (no direct link); 3 (excitatory link with intensity J37 = 0.722); 6 (inhibitory link with intensity Jer = —0.041). 
Fig. (weakly connected unit) excites the units: 1 (inhibitory link with intensity Jio = —1.131); 3,5,8 (no direct link); 
Fig. 13;. 5 receives the excitation from the units: (no direct link); 1 (excitatory link with intensity J51 = 1.015); 5 (no direct 
link); 6 (inhibitory link with intensity J56 = —1.312). 

Several remarks can be made. First, as expected, there are resonance peaks common to all pairs. For example, there 
is a common peak located aX uj — 0.57 with the same width (corresponding to the imaginary part of the corresponding 
pole) Note however that the height can be different (it corresponds to the value of the residue). It is also clear 
from inspection of Figure 13 that there are resonance peaks common to the susceptibilities corresponding to the same 
emitting unit (e.g. 0.45; 0.84; 1.01 in Fig|3t; 0.74; 1.37 if Fig. ^p). There are also peaks that exists only for some pairs 
(e.g. 0.125; 0.2; 2.52 in Fig. Hl; 0.12,0.74,2.83 in Fig. OJd; 0.125,0.70; 1.26 in Fig. |3c). Also, a simple glance at Fig. 
O; shows that characteristics of some of the resonance peaks (namely the frequency corresponding to the maximal 
response and/or the width) of a receiving unit depend on the exciting unit. For example, applying a signal to the 
unit 6 with the frequency lo — 1.26 will induce a strong response of the unit 5, while applying the same signal with 
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the same frequency to the units or 5 will induce a weak response (see section III Bl and Fig. ^]for more details). 
This suggests that a suitable filtering of the global signal arriving at 5 will produce a good signal to noise ratio in the 
case 6 — > 5 while it will be poor in the case ^ 5 or 5 ^ 5 (in this last case the unit does not "feel" the signal even 
if it is applied to itself, because this signal is hidden into the chaotic background). Note finally that some resonance 
peaks are relatively high (~ 20) corresponding to an efficient amplification of a signal with suitable frequency. 

It is also clear from these figures that the intensity of the resonance has no direct connection with the intensity or 
the sign of the coupling and is mainly due to nonlinear effects. For example, there is no direct connection from to 
3 or 5 but nevertheless these units react strongly to a suitable signal injected at unit 0. On the other hand, there 
certainly exists a link between the resonance curves and the topological connectivity of the node: 7 is a topological 
"hub" that sends links to almost every units, and the response curves of the units are rather similar. On the contrary, 
sends a unique link to 1. Consequently, the resonance curves for the other units correspond to "indirect" paths and 
they look different. 

C. Susceptibilities versus correlations. 

As discussed above, Ruelle's theory states that the susceptibility Xij{cj) = xtji^) + Xij{^) where xfj('^) iXiji^)) is 
the stable (unstable) part. Consequently, Xiji'^) contains stable and unstable resonances. Since unstable resonances 
are present in the Fourier transform of the correlation function Cij{uj) it is natural to compare Xij{Lo) and Cij{w). 
An example is given in Fig. 0] As expected, one observes common peaks but there are additional peaks in the 
susceptibility. Note also that the correlation curves are all similar and have the same peaks (only the value of the 
maxima change). 

This figure calls however for an important remark. While the numerical method used for the computation of 
the susceptibilities allows one to have a rather high resolution in frequency {5uj„iin — ^^^d to detect narrow 

resonance peaks, the computation of the correlation function is submitted to much more stringent limitations. 
Indeed, it is well known that, due to the initial condition sensitivity, the maximum time resolution is (assuming an 
attractor with a diameter of order 1) tmax = —j^ lii(''?)j where Ai is the maximum Lyapunov exponent (Ai = 0.153 
in our case) and rj is the round off error ('^ 10"""^^ on a Pentium, in double precision) This gives a tmax of order 
240 corresponding to a frequency resolution SiOmin ^ ^ = 0.026. This is the narrowest width of the resonance 
peaks that one can measure. Using specific libraries in quadruple precision (77 ^ 10~^^) will only divide by 2 the 
frequency resolution. Since this effect is due to initial condition sensitivity in the unstable directions, the stable 
part of the susceptibility is not subjected to these limitations. A consequence of this remark is however that by 
glancing at the example presented here one cannot say whether a resonance peak corresponds to a stable or an 
unstable resonance (except for some striking cases such as w = 0.57). This would require a more careful investigation 
of the corresponding poles, but this is not necessary for the scope of the present work (see for a computation 
of the poles). Indeed, a simple glance at Fig. |3^,b,c reveals a large number of peaks and many of them are not 
present in the correlation curves. For the applications discussed in the following, all what we need to know is that 
the susceptibility contains all resonances (stable and unstable) while the correlation only contains unstable resonances. 

Another observation further underlines the difference between susceptibilities and correlation functions. The 
fiuctuation-dissipation theorem of non-equilibrium statistical physics asserts that the response function to a non- 
equilibrium perturbation can be expressed in terms of a correlation function. Consequently, the information about 
non-equilibrium relaxation is included in the equilibrium fluctuations. It follows in particular that the relaxation time 
towards equilibrium is equal to the decorrelation time (mixing time). 

Consider now figure |31 The Fourier transform of the susceptibility Xij{Lu) is the average response Rij{t) of i to an 
instantaneous kick applied to j aX t = 0. The first row of Fig. [S] shows these responses for the pairs 1 ^ 3, 3 ^ 1 
and 5 — > 5 as well as the corresponding time correlations. Clearly, the coherence time observed in the response is 
substantially longer than the correlation time. Henceforth, the time for returning to equilibrium is different, and in 
our case longer, than the mixing time. This shows that the (average) effect of a kick can be observed on very long 
time scales in spite of chaos. 

One also notes that the response 1 ^ 3 is drastically different from the response 3 — > 1 while correlation functions 
are identical (up to the symmetry Ci3(t) = C3i(— t)). This difference is even more striking when observing the 
Fourier transforms. Since Cij(t) = Cji{—t) the graph of |C'i3(a;)| and \C^i{uj)\ are identical. Consequently, observing 
a resonance peak in the Fourier transform of the correlation function for a pair ij does not tell us "who excites whom" . 
On the other hand, the graph of |xi3(a;)| and |x3i(a;)| displays clearly different peaks. At a frequency uj = 2.52, 3 
excites 1, but 1 does not excite 3. Consequently, the susceptibility provides causal informations contrary to correlation 
functions. The difference comes from the fact that correlation functions deal with the dynamics "on" the attractor. 
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FIG. 4: . Modulus of the correlation functions corresponding to the susceptibilities represented in the Fig. Et,b,c. 

while susceptibilities consider perturbations on the attractor as well as transverse to the attractor. As a matter of 
fact, the presence of stable directions introduces an explicit arrow of time and causality. 

D. Revisiting the notion of "hubs". 

The resonance curves leads us to seriously revisiting the notion of hub. As indicated in Fig. |21 the node 7 is 
a topological hub. However, its ability to propagate a weak periodic signal with frequency lo depends on lo. The 
previous analysis leads then us to propose a notion of "effective" connectivity based on susceptibility curves. For a 
given frequency lo, we plot the modulus of the susceptibility with a representation assigning to each pair i,j 

a circle whose size is proportional to the modulus. Some examples are represented in Fig. |S1 We clearly see in this 
figure that changing the frequency changes the effective network. 

For example, with a frequency lo — 0.125 (Fig. IHt^), the node 1 has a strong ability to transmit signals towards the 
node 5 (namely the response of this unit is high). On the contrary, nodes 5, 6 and ...7 (the topological hub) have weak 
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performances in signal transmission at this frequency. Moreover, one sees that 7 is a bad sender and a bad receiver: 
in this sense it is not a hub at this frequency. With a frequency 0.57 (unstable resonances) the effective network has 
a rather symmetric structure and basically all units respond to this excitation (however with a different amplitude). 
Also, some units present a strong affinity with some others, at a specific frequency. This affinity is however not 
completely specific: the unit 3 "likes" the frequency cu = 0.84 (Fig. Et) whatever is the unit emitting it (but the best 
excitation is provided by unit 7) . Obviously, one also checks that for frequencies that do not correspond to resonances 
(such a,s Lu — 2.33 in Fig. |Sf) the response is essentially inexistent whatever the pair. 

Finally, this figure shows that it is basically possible to excite any unit from any other one in such a way that this 
unit (and possibly a few other but not all the other units) have a maximal response. This can be observed in more 
details in Fig. Ill Dl This is a matrix where the entry i,j (receiver/sender) contains the frequency where the modulus 
of the susceptibility is maximum (first value) and the value of this maximum (second value). Clearly, some units are 
more "excitable" than others (such as 5). 

All these effects are due to a combination of topology and dynamics and they cannot be read in the connectivity 
matrix J. 



(3.1,5.84) (0.572,8.91) (0.756,5.97) (3.14,10.6) (3.05,6.8) (3.03,6.3) (0.724,6.71) (3.14,7.85) (3.14,8.7) 



(0.563, 9.77) 
(1.35, 4.45) 
(0.563, 15.4) 
(1.35, 4.6) 
(0.36, 21.6) 
(0.141, 6) 
(0.563, 6) 
(3.13, 19.4) 



(0.569, 22.2) 
(0.569, 8.14) 
(0.569, 35.3) 
(0.569, 5.32) 
(0.127, 28.2) 
(0.569, 10.1) 
(0.569, 14.1) 
(3.12, 14.9) 



(0.795, 11) 

(1.27, 7.13) 
(0.795, 21.9) 

(1.31, 7.67) 
(0.385, 23.5) 

(1.31, 7.58) 
(0.795, 7.86) 

(3.02, 16.9) 



(0.479, 10.7) 
(0.719, 4.42) 
(0.463, 20.9) 
(0.742, 4.04) 
(0.437, 27.8) 
(0.437, 8.37) 
(0.479, 8.9) 
(3.14, 32) 



(0.577, 12.4) 

(1.28, 8.73) 
(0.799, 19.8) 

(1.28, 10.5) 
(0.138, 17.6) 

(1.28, 9.33) 
(0.563, 7.53) 

(3.04, 21.1) 



(0.373, 5.55) 
(1.39, 5.05) 
(0.816, 11.5) 
(1.39, 5.22) 
(0.364, 17.6) 
(1.39, 4.53) 
(0.399, 4.37) 
(3.03, 20.2) 



(0.68, 13.4) 
(1.41, 7.83) 
(0.71, 19.8) 
(1.26, 8.3) 
(1.26, 15.5) 
(1.26, 9.09) 
(0.71, 7.91) 
(3.02, 13.4) 



(0 



569, 14.5) 



844, 7.07) 
844, 22.1) 
844, 6.88) 
569, 19.1) 
854, 6.79) 
(0.569, 10) 
(3.14, 26.5) 



(0.577, 6.44) 
(1.3, 4.5) 

(0.482, 11.3) 
(1.3, 5.53) 
(0.129, 16) 
(1.3, 5.23) 
(0.5, 4.61) 
(3.14, 25.8) 



Matrix where the entry i,j (receiver /sender) contains the frequency where the modulus of the susceptibility is maximum (first value) and 

the value of this maximum (second value) 



[Extrapolating further the possibilities suggested by these figures, one may imagine to apply at node 3 a superpo- 
sition of signals, with amplitude modulation, but with a different carrier frequency (e.g. toi = 0.125 and lu2 = 2.33), 
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FIG. 6; EfTective connectivity for uj = 0.125; 0.57; 0.84; 1.0; 1.26; 2.33. 



such that 5 and 8 respond simultaneously to their own resonance frequency. However, this operation requires to be 
strictly in the linear response regime (small e).] 



E. Effect of e. 



A linear response theory assumes that e is small enough, so that Xij^ corresponding to the first order term in 
the e expansion of < Su >, is independent of e. Henceforth, to check that the value e — 10~^ chosen in our 
simulations is sufficiently small, we have to verify that multiplying or dividing e by some (small factor) does not 
change the susceptibility. Actually, later on we will also be interested in larger values of e where the nonlinear terms 
in the e expansion are non negligible (see section III G!|l . Although this regime brings the system out of the linear 
response setting, it provides interesting stability properties for amplitude modulation. However, it is well known that 
nonlinearities change the resonance structure. Consequently, we have investigated the influence of increasing e on the 
susceptibilities. Some examples are depicted in Fig. [7|for the pair 3 — > 1 

One remarks that the susceptibility is stable in the range [5 x 10^''; 2. x 10^^] (note however that the signal is more 
noisy when e is weaker, explaining the larger fluctuations for e = 5 x 10^**). Increasing e further leads to distortions 
in the resonance curve (see section iHTTjl . 



F. Amplitude modulation. 



The existence of strong amplitude specific resonances opens up the possibility for transmitting a signal carrying 
information from a node to a target node, in such a way that, with a suitable filtering of the chaotic background, the 
initial signal can be recovered. For this, one may use amplitude modulation where the characteristic time scale for 
the modulation is sufficiently long. More precisely, one performs the average (0) with a sliding time window whose 
width is sufficiently large to have small fluctuations but remains sufficiently small compared to the characteristic time 
for the modulation. 

To illustrate this point we have superimposed a signal with periodic amplitude modulation, ^{t) — 
ecos{ujMt)sin{ojot) where u>o is a resonance frequency and ojm the frequency of the amplitude modulation. 
Note that, in the case e = 10~^ the variations of the signal amplitude stay within the limits where linear response 
theory applies. However, for such a weak signal amplitude, the signal/noise ratio is very large and we had to perform 
the average (UJ over a time windows of width T = 10^. This imposes strong constraints on the modulation frequency, 
which has to be smaller than ^ to have a correct sampling of the signal. The simulations have been done with 
lum — 2 ii23r ^ 2.97.10"^. The (arbitrary) non integer factor 2.1123 has been introduced to avoid commensurability 
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FIG. 7: Effect of increasing e for the susceptibility xi3 for e G [5 x 10 5 x 10 



between the frequency corresponding to the shding window (^) and lom- 



We have first considered (Fig. |SJ| the case 7^3 with a resonance frequency lu — 0.57 (see Fig|31i). One clearly sees 
that the signal can be recovered by the averaging procedure ^ . Note also that one gets an effective amplification by 
a factor ~ 20 in agreement with the resonance curve ISt- Consequently, and contrary to conventional intuition about 
chaotic systems, a weak signal superimposed upon a chaotic background can be recovered provided one performs a 
suitable average over the chaotic dynamics. In some sense, this idea is already contained in Boltzmann's work where 
macroscopic observable values are obtained by averaging over the microscopic molecular chaos. 

We have then investigated the possibility of sending a signal from a unit to some target by suitably selecting the 
frequency. In section Hi BI we have given the example of exciting 5 with a frequency 0.125 or 1.26. In the first case, it is 
expected that a signal emitted from or 1 will be correctly received by 5 while the same signal emitted from 5 or 6 will 
not be distinguished from the chaotic background. This is verified in Fig. |S| The first column represents the response 
after performing the average Q on the perturbed system u. The right column represents the same average performed 
on the unperturbed system (without signal) u. It is clear that the signal emitted by the units 0, 1 is correctly recovered 
by unit 5 (with however some distortions) while there is no clear difference between the perturbed and unperturbed 
cases when the units 5,6 are emitting the same signal. Note that the average corresponding to each pair have been 
performed for different initial conditions. This explains why the figures in the right column are different. 

The case w = 1.26 is presented in the Fig. [TUl 



G. nonlinear regime. 



The main drawback of the previous examples is the weakness of the signal. In order to have an efficient elimination 
of the chaotic background one needs to average over a long time window, limiting de facto the modulation frequencies, 
and, even so, the decoded signal is not completely satisfactory. It is then reasonable to increase the amplitude of 
the signal. But first one has then to check that the resulting dynamics remains chaotic. Indeed, too large a signal 
will irretrievably "kill" the background. Even when e is weak enough so that one may still consider the signal as 
a perturbation, increasing the signal/noise ratio can drive the system outside the linear response regime. Indeed, 
as suggested in section fll El the susceptibility curves are modified when e is larger than 5.10^^. In this section we 
investigate this effect more carefully. 

First we note that an explicit formula for nonlinear corrections have been worked out by Ruelle in 15]. However, 
it is hardly tractable, even in the case of model where the linear response has a simple form. We used then 
the numerical computation Q for e = 10^^. In Fig. Illb .b.c we have represented the susceptibility curves for the 
same cases as in Fig. |2K,b,c, section III Bl One observes sharper resonance peaks. This corresponds to having poles 
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FIG. 8; Excitation of 3 by 7 with a signal f (t) — ecos{uj Mt)sin{u)ot) with a fundamental frequency ujo — 0.7 and a modulation 
frequency ujm ~ 2.97.10~®. The column on the left represents, the unperturbed local field U3{t), the perturbed local field U3{t), 
the difference between the two local fields Su{t) = U3{t) —us{t), the time average Q with a time window T — 10® for U3{t) (4th 
row) and U3{t) (5th row), respectively. The signal is represented in red, with a magnification by a factor 10, in order to see it 
on the graph. The second column represents the same quantities with representation u{t + 1) vs u{t) giving a two dimensional 
projection of the dynamics in the phase space. . 
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FIG. 9: First column. Response of unit 5 respectively to an excitation of the units 0,1,5 and 6, with amplitude modulation. 
The fundamental frequency is loq = 0.125 and the modulation frequency is com — 2.9710~®. The second column represents the 
same average performed on the unperturbed dynamics. Note that the average corresponding to each pair have been performed 
for different initial conditions. 



approaching the real axis when increasing the strength of the periodic forcing. Certainly, for sufBciently large e and 
for specific resonant frequencies, one expects the dynamics to "lock" on a periodic orbit with the effect of "killing" 
the chaotic regime. This is however not yet the situation for the value of e investigated here, as verified here (see e.g. 

Fig. 113. 
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FIG. 10: Response of unit 5 to an excitation of the units 0, 1, 5, 6 with amplitude modulation. The fundamental frequency is 
Luo — 1.26 and the modulation frequency is ujm ~ 2.9710"^. 



As an example, we have excited the unit with a frequency co = 0.32, corresponding to a sharp resonance and with 
an amplitude modulation frequency llIq — 2.9710^^ (note that this frequency is ten times higher than the previous 
one. We were then able to shrink the sliding window by a factor 10). The response of the 9 units is plotted in Fig. 
El The signal/noise ratio is substantially better. We also observe that several units respond, but the more accurate 
response corresponds to the unit 5. 

How does the perturbed evolution of this unit look like? The unperturbed and perturbed trajectories are plotted 
in Fig. 1131 One does not see any difference. In particular the perturbed dynamics is still chaotic. 

Assume now that there is a user at node 5, observing the dynamics. Without filtering, he does not notice any 
difference between the system with and without signal. But, if he knows the carrier frequency, he is able to recover 
a signal emitted from the unit out of the chaotic background. This suggests a way to encode hidden information 
in a chaotic signal. There is in fact a little bit more. The same user located at 2, 3 or 8 will not be able to recover 
a sufficiently good signal. In this sense, the nonlinearity allows us to send the signal to specific targets by a suitable 
choice of the frequency modulation. 
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FIG. 11: . Resonance curves in the same cases as in Fig. |3i,b,c, for e = 10" 



III. DISCUSSION. 



In this paper, we have exhibited some non-intuitive aspects of networks with nonhnear relays and chaotic dynamics. 
These effects were predicted on the basis of general theoretical arguments, analyzing the linear response of such systems 
to an excitation by a signal injected at some place in the network. We have in particular argued that saturation effects 
in the nonlinear transfer function of the node induces the presence of resonances, the stable ones, that are not present 
in the correlation functions. These resonances, corresponding to the response of the system to out-of-equilibrium 
perturbations, can be used to produce unexpected results in chaotic systems, such as the transmission of a signal 
with amplitude modulation from a node to some target. Though the signal is basically weak, it can be recovered by 
a suitable averaging procedure, in spite of chaos. Moreover, thanks to chaos, this recovery can only be performed if 
the receiving user knows the carrier frequency and if he is located at the right node. Note that this transmission is 
robust with respect to noise, as we checked. 

Furthermore, it would be most interesting to realise an implementation of this scheme on "real" experimental 
chaotic networks. For example, the frequency dependent averaging of eq. Q could be implemented with a "lock-in" 
amplifier [T6l|. 

We have presented an example supporting these conjectures. We have briefly discussed in section |n] the genericity 
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FIG. 12: Response of all units to an excitation of the units with amplitude modulation and e 
frequency is a; = 0.32 and the modulation frequency is loo = 2.9710~^. 



0.01. The fundamental 




FIG. 13: Projection of the trajectories U5{t) (unperturbed system) and U5{t) (perturbed system with amplitude modulation, 
e — 0.01, ui = 0.32 and modulation frequency is luq = 2.9710~^). 



of this example. But what about the genericity of the model itself? As discussed in the introduction this model 
contains some essential features such as the competition of excitation/inhibition, the asymmetry of the interactions, 
and the saturation of the transfer functions that are basically present in biological networks or in some communication 
networks. These features generically produces chaos in systems like provided that the nonlinearity is sufficiently 
large (see for a recent review). However, one does not necessarily have chaos in "real" networks. It might 
indeed well be that biological networks, for example, are often closer to intermittency than to chaos and closer to 
bifurcation points than to structurally stable hyperbolic systems (see In this case, the application of the 

methods developed here may lead to fundamental questions such as: do we still have a linear response theory in 
this case ? As discussed in the paper, when one approaches a structurally unstable point (bifurcation point) the 
susceptibility may diverge, as it does in physical systems at a second order phase transition. Note however, that the 
the way the susceptibility diverges is already a crucial information. Actually, in systems undergoing a second order 
phase transition, the divergence occurs if one takes the thermodynamic limit, but physical systems are finite. In the 
same way, the divergence of our susceptibility may occur when one takes the infinite time limit, but real networks 
are investigated on finite times. This question deserves therefore further investigations. 
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Nevertheless, the simple fact that we have been able to produce one example of the effects theoretically predicted 
for such networks, obliges, in our opinion, the community to review some a priori. Though it is helpful to investigate 
the topological properties of complex networks (small world, scale free graphs and so on) it is in no way sufficient 
for characterizing the ability of transmission of such networks with active nodes. Moreover, the dynamics of signal 
propagation is not a superposition of the graph properties and of the local input/output dynamics. There is a complex 
nonlinear feedback between the two (as revealed for example in eq. ©) and one has to study it as a whole. This may 
require the development of new tools such as the linear response theory presented here. The development of such tools 
opens new perspectives and suggest new applications for communication networks, as well as a better understanding 
of biological networks. 
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